1. Field of the Invention
This invention relates to a focus detecting device for use in an optical apparatus such as a camera, and particularly to a signal processing method for focus detection.
2. Related Background Art
As one focus detecting device for single lens reflex cameras, there is well known a device in which the pupil of a photo-taking lens is divided into two areas and the displacement of the relative position of two object images formed by light fluxes passed through the divided pupil areas is observed to thereby detect the focus state of the photo-taking lens. The principle of this device will hereinafter be described briefly with reference to FIG. 10 of the accompanying drawings.
In FIG. 10, a field lens 2 is disposed coaxially with a photo-taking lens 1 to be focus-detected, and two secondary imaging lenses 3a and 3b are juxtaposed rearwardly of the field lens 2, and light receiving photoelectric conversion sensor arrays 4a and 4b are disposed rearwardly of the secondary imaging lenses 3a and 3b, respectively. Designated by 5a and 5b are stops provided near the secondary imaging lenses. The field lens 2 substantially images the exit pupil of the photo-taking lens 1 on the pupil planes of the two secondary imaging lenses 3a and 3b. As a result, light fluxes entering the secondary imaging lenses 3a and 3b emerge from areas of equal dimensions on the exit surface of the photo-taking lens 1 which correspond to the secondary imaging lenses 3a and 3b and do not overlay, each other.
When aerial images formed near the field lens 2 are re-imaged on the surfaces of the sensor arrays 4a and 4b by the secondary imaging lenses 3a and 3b, the re-imaged two images change their positions on the basis of the difference between the positions in the direction of the optical axis at which said aerial images are formed.
FIG. 11 of the accompanying drawings shows the manner in which this phenomenon occurs. Depending on whether, with the in-focus state of FIG. 11A as the center, the so-called focus state is front focus or rear focus as shown in FIG. 11B or 11C, the two images formed on the surfaces of the sensor arrays 4a and 4b move in opposite directions on said surfaces. If this image intensity distribution is photoelectrically converted and an electrical process is effected on the obtained photoelectric conversion signal to thereby detect the relative positional deviation of said two images, the focus detection of the photo-taking lens can be accomplished.
Numerous methods of detecting the amount of deviation of said two images have been proposed, and the most popular one of them is a method of finding a correlation which shifts the two photoelectric conversion image signals relative to each other, and regards the shift position highest in the correlation as the amount of image deviation.
For example, various methods are disclosed in Japanese Laid-Open Patent Application No. 56-75607 (hereinafter referred to as "Prior Art Example 1"), Japanese Laid-Open Patent Application No. 57-45510 (hereinafter referred to as "Prior Art Example 2"), Japanese Laid-Open Patent Application No. 60-247211 (hereinafter referred to as "Prior Art Example 3"), and applicants' Japanese Laid-Open Patent Application No. 58-142306 or No. 59-107313 (hereinafter referred to as "Prior Art Example 4").
The techniques of correlation calculation disclosed in the above-mentioned prior art examples are characterized by the following procedures:
(1) the operator of correlation calculation; PA1 (2) the manner of finding the external value of the amount of correlation; and PA1 (3) a method of shifting the correlation calculation.
The operator will first be described.
In Prior Art Example 1, Prior Art Example 2 and Prior Art Example 3, the amount of correlation is calculated with "the absolute value of the difference between two corresponding signals" or "the power of the absolute value of the difference between two corresponding signals" used as the operator of a correlation function.
In prior Art Example 4, "a greater (or smaller) value of two corresponding signals" is used as the operator of a correlation function.
The meaning of the operator will now be described briefly.
When two object image signal systems are defined as IA(i) and IB(i), the operator called "the absolute value of the difference between two corresponding signals" (hereinafter referred to as "the absolute value of the difference") calculates the amount of correlation COR(K) in shift K in accordance with the following equation (1): ##EQU1## On the other hand, the operator called "a greater value of two corresponding signals" or "a smaller value of two corresponding signals" (hereinafter referred to as "a greater value" or "a smaller value") calculates the amount of correlation COR(K) in accordance with the following equations (2) and (3): ##EQU2## In the above equations, Max{x, y} and Min{x, y} are operators which extract the greater one and the smaller one, respectively, of real numbers x and y.
When the amount of correlation is calculated in accordance with the above-mentioned equations, an amount of shift which is greatest in correlation can be determined from the amount of correlation obtained for each shift. This amount of shift corresponds to the amount of image deviation of two object images, and from this, the focus state (the defocus amount) of the photo-taking lens can be detected.
If as described above, the amount of shift which is greatest in correlation is simply regarded as the amount of image deviation, the detectable amount of image deviation is only obtained for one shift of the object signal system, i.e., at the integer unit of one picture element. If this is converted into a defocus amount, the resolving power thereof will become relatively rough and will be of an insufficient level in practice.
The aforementioned prior art examples disclose techniques of detecting an amount of image deviation which is greatest in correlation with a resolving power of a decimal unit finer than the integer unit of one picture element, and those techniques are classified broadly into two kinds.
One of the two kinds is a technique of finding an extreme value of correlation of a decimal unit from an amount of correlation of an integer shift unit directly by interpolation (hereinafter referred to as "the direct interpolation technique"). The other technique is a technique of interpolating and finding a shift of a decimal unit in which a linear derived function, i.e., the differential value of the amount of correlation (although actually it is a difference value because the amount of correlation is a discrete amount of a shift unit) becomes zero (hereinafter referred to as "the differential interpolation technique"). This is based on the principle that when the linear derived function becomes zero, the original function thereof assumes an extreme value.
In "the direct interpolation technique", an amount of shift for which the amount of correlation is expected to become smallest is interpolated and found from the amounts of correlation at three points, i.e., a shift in which the amount of correlation calculated at each shift becomes smallest and the shifts before and after said shift, at a measure finer than the shift unit, and this is regarded as the amount of image deviation. The aforementioned Prior Art Example 3 corresponds to this, and its specific method is disclosed in detail in Prior Art Example 3.
On the other hand, in "the differential interpolation technique", the difference amount of the amount of correlation is calculated in advance at each shift and a shift in which the sign of the difference amount varies is detected, and from the difference amount between that shift and its neighboring shifts, an amount of shift in which the difference amount is expected to become zero, that is, the amount of correlation is expected to assume an extreme value, is interpolated and found also at a measure finer than the shift unit.
For example, the correlation difference amount at the shift K when the operator is "the absolute value of difference" is calculated in accordance with the following equation (4): EQU DIF(K)=.SIGMA..vertline.IA(i) - IB(i+K+1).vertline.-.SIGMA..vertline.IA(i+1) - IB(i+K).vertline. (4)
The extreme value detecting techniques of the aforementioned Prior Art Example 1, Prior Art Example 2 and Prior Art Example 4 correspond to this, and their specific method is disclosed in detail in Prior Art Example 2.
Description will now be made of a shifting method of correlation calculation.
The setting method of applicants' Japanese Laid-Open Patent Application No. 56-50315 (hereinafter referred to as "Prior Art Example 5") will hereinafter be described with reference to FIG. 12 of the accompanying drawings.
In FIG. 12, the reference characters 10a and 10b indicate the intensity distributions of two object images on the surfaces of the two sensor arrays 4a and 4b in FIG. 10, and this example represents a case where simple geometrical edge images deviate by 0.5 picture elements from each other. Each sensor is comprised of 8 picture elements, the number of signals of the image signal system is also shown as 8 picture elements. Arrows 12a and 12b indicate calculation areas for correlation calculation, and when the shift k of correlation calculation is, e.g. 4, as shown in FIG. 12, the correlation among signals of 4 picture elements on the left side is calculated by the signal 10a and the correlation among signals of 4 picture elements on the right side is calculated by the signal 10b. When the shift k is 2, the calculation areas are symmetrically shifted by one picture element each, and the correlation among image signals of 4 picture elements is likewise calculated. Likewise, with the number of calculation picture elements kept fixed, the calculation areas are varied as shown in conformity with the shift. The sign of the shift k corresponds to the so-called front focus and rear focus.
The method of shifting the calculation areas of two image signals symmetrically and mutually in conformity with the correlation shift will hereinafter be referred to as the "mutual shift".
In the "mutual shift" method, two image signals are shifted at a time and therefore, unavoidably, the shift of correlation is a 2-shift unit. Accordingly, if the number of signals of the image signal system is 8 picture elements and the number of calculation picture elements is 4, the shift k which can be calculated is -4, -2, 0, 2 and 4.
FIG. 15 of the accompanying drawings is a graph in which the amount of correlation calculated by the "mutual shift" is plotted by the use of the "absolute value of the difference" as the operator. C(-4), C(-2), C(0), C(2) and C(4) represent the amounts of correlation in the shift k=-4, -2, 0, 2, 4, respectively. To detect an extreme value of correlation of a decimal unit from this plot, the extreme value of correlation can be interpolated from the three points at the amounts of correlation C(-2), C(0) and C(2) by the use of "the direct interpolation technique", as indicated by broken line in FIG. 15 and at the same time, an amount of shift kp which gives that extreme value can be found. A method of finding the extreme value of correlation of a decimal unit from the amounts of correlation of three points is disclosed in detail in "Prior Art Example 3" and therefore need not be described herein.
FIG. 15 is a graph in which the amount of correlation is plotted by the use of an operator which is the "absolute value of difference" in order to illustrate the "mutual shift". In Prior Art Example 5, the "mutual shift" is used as the shifting method, but how to find the extreme value of correlation is not specifically shown.
The shifting method in Prior Art Example 3 will now be described with reference to FIG. 13 of the accompanying drawings.
In FIG. 13, as in FIG. 12, the reference characters 10a and 10b indicate the intensity distributions of two object images, and arrows 12a and 12b indicate calculation areas for correlation calculation.
The feature of the shifting method in Prior Art Example 3 is that two object images are characterized as a "fiducial portion" and a "reference portion" and correlation calculation is executed with the fiducial portion remaining fixed and with the image signal of the reference portion shifted.
As shown, as long as the shift k is between -2 and 2, the calculation area 12a for the image signal 10a of the fiducial portion is fixed and only the calculation area 12b for the image signal 10b of the reference portion moves. In this shifting method, only one of the image signals moves and therefore, the correlation calculation for each shift is possible unlike the case of the aforedescribed "mutual shift" of Prior Art Example 5.
When the shift k=-2 or 2, the calculation area 12b for the image signal 10b of the reference portion strikes against the end of the whole signal area and therefore cannot move any more. So, in order to calculate the shift k=-3 or greater, such a contrivance is made that the calculation area 12a for the image signal 10a of the fiducial portion which should originally be fixed is moved by an amount corresponding to two picture elements and the shift k=-3 of two images is relatively realized. This also holds true of the shift k=3 and so on.
The shifting method in which like this, the calculation area for one image signal is fixed and only the calculation area for the other image signal is moved will hereinafter be referred to as the "one image shift".
FIG. 16 of the accompanying drawings shows a graph in which the amounts of correlation calculated by the use of the operator, "the absolute value of difference" and the shifting method, the "one image shift" of Prior Art Example 3 are plotted.
In FIG. 16, C(-4) to C(4) represent the amounts of correlation at the shift k=-4 to 4, respectively.
To detect the extreme value of correlation of a decimal unit from this plot, a minimum value of correlation can be interpolated from three points, i.e. , the amounts of correlation C(-1), C(0) and C(1) or C(0), C(1) and C(2), by the use of "the direct interpolation technique" as indicated by broken line and at the same time, an amount of shift kp which gives that minimum value can be found.
Both Prior Art Example 5 and Prior Art Example 3 find the amount of shift of a decimal unit which gives the extreme value of correlation, but depending on the difference in the shifting method, in Prior Art Example 5, use is made of three amounts of correlation spaced apart from one another by two shifts each, whereas in Prior Art Example 3, use is made of three amounts of correlation spaced apart from one another by one shift each.
The shifting method and how to find the extreme value of correlation in Prior Art Example 2 will now be described with reference to FIG. 14A of the accompanying drawings.
In FIG. 14A, as; in FIGS. 12 and 13, the reference characters 10a and 10b indicate the intensity distributions of two object images, and arrows 12a and 12b indicate calculation areas for correlation calculation.
The feature of the shifting method of "Prior" Art Example 2 is that as shown, at the shift k=0, the whole signal area of image signals 10a and 10b is the object of calculation and as shift is made from k=0, the calculation range is symmetrically decreased from one end of the calculation area.
This shifting method resembles the "mutual shift" of Prior Art Example 5 but in the "mutual shift, the absolute positions of the calculation areas of two image signals are mutually and symmetrically changed with the number of calculation picture elements remaining fixed as the shift is made. Thus, the change of the relative position of the two calculation areas is effected in a two-shift unit.
In contrast, in Prior Art Example 2 (FIG. 14A), as a shift is made, the number of calculation picture elements changes and the absolute position of the calculation area for one of two image signals is not changed by the positive or the negative sign of the shift k. Accordingly, two calculation areas are effected in a one-shift unit.
The shifting method whereby like this, only one of the calculation areas is moved while the calculation picture elements of two image signals are decreased will hereinafter be referred to as "calculation number variable shift".
What must be specified here is that in Prior Art Example 2, correlation calculation is effected in the calculation areas themselves indicated by arrows 12a and 12b in FIG. 14A. FIG. 14A illustrates the difference between Prior Art Example 2 and the other prior art examples. Prior Art Example 2 actually uses "the differential interpolation technique" as the method of detecting the extreme value of correlation and therefore is designed such that, as shown in FIG. 14B of the accompanying drawings, the calculation areas are decreased by one picture element each at a certain shift and accordingly, the amounts of calculation are calculated in two pairs of calculation areas (15a, 15b) and (16a, 16b) spaced apart from each other by an amount corresponding to .+-.1 shift about that shift and the difference therebetween is calculated.
FIG. 17 of the accompanying drawings is a graph in which the amounts of correlation difference D(-4) - D(4) at each shift are plotted. The operator is "the absolute value of difference".
Since the object images mentioned as the examples are originally deviated from each other by 0.5 picture element, the amount of correlation should assume an extreme value at the shift k=0.5. As is apparent from FIG. 17, the amount of correlation difference has its sign changed at the shift k=0 and the shift k=1, and this means that the amount of correlation difference is zero between the shift k=0 and the shift k=1, that is, there exists an amount of shift of a decimal unit in which the amount of correlation which is an original function gives an extreme value. So, in Prior Art Example 2, two points, i.e., the amounts of correlation difference D(0) and D(1), are interpolated to thereby find an amount of shift kp in which the amount of correlation difference is estimated as zero. The interpolation method of finding an extreme value of correlation of a decimal unit from the amounts of correlation difference of two points is disclosed in detail in Prior Art Example 2 and therefore need not be described herein.
As regards the method of detecting image deviation disclosed in applicants' Prior Art Example 4, the interpolation method for the extreme value of correlation and the shifting method are "the differential interpolation technique" and "the calculation number variable shift" identical to Prior Art Example 2, but are techniques using "a greater value"]or "a smaller value" as the operator.
FIG. 18 of the accompanying drawings shows a graph in which the amounts of correlation difference by Prior Art Example 4 are plotted. The amounts of correlation difference E(-4) - E(4) represent a case where "a greater value" is used as the operator, and the amounts of correlation difference F-4) - F(4) represent a case where "a smaller value" is used as the operator. The method of finding an extreme value of correlation kp of a shift unit or less may be the same as Prior Art Example 2.
As previously described, in Prior Art Example 2, the extreme value of correlation is detected by the use of the amount of difference and therefore, it is permitted to vary the number of calculation picture elements in conformity with shift. On the other hand, in the case of "the direct interpolation technique" in which the extreme value of correlation is directly detected, if the number of calculation picture elements at each shift is varied, there may arise a case where a right extreme value cannot be detected because the amount of information contributing to each amount of correlation is not uniform. That is, "the direct interpolation technique" which directly interpolates the extreme value from the amount of correlation holds good because it tacitly assumes that the amount of correlation varies at a predetermined rate in conformity with shift, and if the number of picture elements used in the calculation of the amount of correlation is varied by shift, the amount of correlation will be varied by two factors, i.e., shift and the number of calculation picture elements, and interpolation calculation will not hold good.
Where as in Prior Art Example 2, the "calculation number variable/shift" is used, it is combined with "the differential interpolation technique" and as shown in FIG. 14B, the number of calculation picture elements for the amounts of correlation spaced apart from each other by two shifts which effects difference calculation is made constant for each shift.
The image deviation detecting techniques of the prior art described hitherto may be summed up thus:
1. As the operator of correlation calculation:
(1.1) "the absolute value of difference" (Prior Art Examples 1, 2 and 3); and
(1.2) "a greater value (or a smaller value)" (Prior Art Example 4);
2. As the interpolation method for the extreme value of correlation:
(2.1) "the direct interpolation technique" (Prior Art Example 3); and
(2.2) "the differential interpolation technique" (Prior Art Examples 1, 2 and 3); and
3. As the correlation calculation shifting method:
(3.1) "mutual shift" (Prior Art Example 5);
(3.2) "one image shift" (Prior Art Example 3); and
(3.3) "calculation number variable shift" (Prior Art Examples 2 and 4).
However, it is better that the "calculation number variable shift" mentioned under item (3.3) above not be used in combination with "the direct interpolation technique" mentioned under item (2.1) above.
Now, as described above, various methods of detecting image deviation have been proposed, but those methods give rise to differences in performance depending on the object image observed.
For a geometrical object image entirely free of noise, a right amount of image deviation can be detected even if any technique is used. However, if random noise from a photoelectric conversion sensor or low-frequency noise such as the ghost of an optical system mixes with the object image signal, there will arise a difference in S/N which depends on the object image, depending on the difference between techniques.
As an example, in the case of a pattern in which the object image observed is generally flat like a so-called edge pattern and has only partially a variation in luminance, if a portion having that variation in luminance exists in the end portion of the calculation area of correlation calculation, a method using "a greater value (or a smaller value)" is superior in S/N to an image deviation detecting method using "the absolute value of difference" as the operator. The reason is described in detail in applicants' Japanese Laid-Open Patent Application No. 60-101513 (hereinafter referred to as "Prior Art Example 6").
Also, when low-frequency noise such as the ghost of an optical system mixes with the object image signal, there is generally the tendency that the error is smaller in "the direct interpolation technique" of directly finding an extreme value from an amount of correlation than in "the differential interpolation technique" of finding an extreme value from an amount of correlation difference as the interpolation method for an extreme value of correlation, although it depends on the object. This is considered to be attributable chiefly to the amount of correlation of how many points being used for interpolation calculation. "The differential interpolation technique" effects interpolation calculation by the use of the amount of correlation difference of two points but substantially it effects interpolation calculation by the use of the amount of correlation of four points because the amount of difference is calculated from the amount of correlation of two points as shown in equation (4). On the other hand, "the direct interpolation technique" uses the amount of correlation of three points. Thus, if the amount of correlation of each shift is fluctuated by noise, "the differential interpolation technique" using a greater number of amounts of correlation is more liable to be affected by noise.
Also, as the shift method for correlation calculation, the "mutual shift" can be said to be more disadvantageous in S/N than the other methods. In the case of the "mutual shift", as previously described, the amount of correlation can be obtained only at two-shift unit, while in the other two methods, i.e., the "one image shift" and the "calculation number variable shift", the amount of correlation can be obtained at one-shift unit. Thus, in the interpolation calculation of the "mutual shift", calculation is effected for the position of the extreme value of correlation to be found, by the use of the .amount of correlation at a shift farther in distance than in the other methods. It is readily anticipated that the error of interpolation calculation will be enlarged if the distance between the coordinates of the amounts of correlation used for interpolation calculation becomes longer.
Summing up what has been hitherto described about the relative superiority of the image deviation detecting methods, by procedures, it can be said that as regards the operator, "a greater value (or a smaller value)" is generally superior in S/N, as regards the interpolation technique for the extreme value of correlation, "the direct interpolation technique" is generally superior in S/N, and as regards the shifting method for correlation calculation, except the "mutual shift", the "one image shift", if combined with "the direct interpolation technique", is generally superior in S/N.
However, an image deviation detecting method comprising a combination of the above-described preferable procedures cannot detect a right amount of image deviation. The plot of the amount of correlation by this combination is shown in FIG. 19 of the accompanying drawings. "A greater value (or a smaller value)" is used as the operator, "the direct interpolation technique" is used as the interpolation technique for the extreme value of correlation, and the "one image shift" is used as the shifting method. A case where a geometrical edge image deviates by 0.5 picture element as in the examples hitherto described is supposed as the object image signal.
The amounts of correlation A(-4) - A(4) represent a case where "a greater value" is used as the operator, and the amounts of correlation B(-4) - B(4) represent a case where "a smaller value" is used as the operator. As can be seen from FIG. 19, whichever of said two amounts of correlation is used, the position is not shown at 0.5 picture element which is the original amount of image deviation, that is, the extreme value is not shown at the shift k=0.5, and correct image deviation detection is impossible.
However, if the object image is of such a geometrical bar pattern as shown in FIG. 20A or 20B of the accompanying drawings, correct image deviation detection will became possible if "a greater value (or a smaller value)" is used as the operator. The plots of the amounts of correlation in these cases are shown in FIGS. 21A and 21B of the accompanying drawings. The amounts of correlation A(-4) - A(4) and the amounts of correlation B(-4) - B(4) are the amounts of correlation when the operator is "a greater value" and "a smaller value", respectively, and a case where as in the examples hitherto described, two images deviate from each other by 0.5 picture element is supposed as the object image. In this case, whichever of said two amounts of correlation is used, the amount of shift kp of the extreme value of correlation represents 0.5 picture element which is the original amount of image deviation, and a right extreme value can be detected.
The reason why the operator which is "a greater value (or a smaller value )" has such a nature can be considered as follows.
The correlation function is a function of which amount of correlation becomes smallest (or greatest) at a shift position whereat two signal systems are most similar to each other and the amount of correlation increases (or decreases) away from that shift position. However, if with "a greater value" as the operator and moreover with the calculation area of one image fixed, the amount of correlation of a "bilateral asymmetrical" image signal like an edge image is calculated, there occurs a situation in which if the sign of the shift is positive, the amount of correlation thereof varies in conformity with the shift, but if the sign of the shift is negative, the amount of correlation does not vary even if the shift is changed. That is, when the edge portion of one image comes into an "envelope" made by the luminance signal of a fixed edge image, even if the image that has come in that state is shifted, the operator which is "a greater value" will always only extract a predetermined signal along the envelope of the fixed image and no variation will appear in the amount of correlation.
In the case of the "mutual shift", the calculation area is always varying without one image being fixed and therefore, the state as described above does not occur. Accordingly, irrespective of the sign of the shift, the amount of correlation varies in conformity with the shift and the detection of the extreme value of correlation becomes possible.
After all, the operator which is "a greater value sum (or a smaller value sum) of two corresponding signals" is an operator which, depending on the object, cannot detect a right amount of image deviation if the calculation area of one of the image signals is combined with fixed correlation calculation.